Papers with syntactic tasks
A logical-based corpus for cross-lingual evaluation (D19-61)
Copied to clipboard
| Challenge: | a recent study shows that deep learning models can be used to solve textual inference tasks using simple linguistic patterns. |
| Approach: | They propose a set of syntactic tasks focused on contradiction detection that exploit linguistic patterns. |
| Outcome: | The proposed tasks can be implemented in English and Portuguese. |
Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks (2020.tacl-1)
Copied to clipboard
| Challenge: | Inductive biases can arise from any aspect of the model architecture, study finds . we investigate which architectural factors affect how models generalize . |
| Approach: | They investigate which architectural factors affect generalization behavior of neural network models . they use English question formation and English tense reinflection as test cases . |
| Outcome: | The findings suggest that human-like generalization requires architectural syntactic structure. |
Attention on Multiword Expressions: A Multilingual Study of BERT-based Models with Regard to Idiomaticity and Microsyntax (2025.findings-naacl)
Copied to clipboard
| Challenge: | Specifically, models fine-tuned on semantic tasks tend to distribute attention to idiomatic expressions more evenly across layers. |
| Approach: | They analyze attention patterns of encoder-only models towards two distinct types of Multiword Expressions (MWEs) idioms present challenges in semantic non-compositionality, while MSUs demonstrate unconventional syntactic behavior that does not conform to standard grammatical categorizations. |
| Outcome: | The proposed models show that fine-tuned models allocate attention to idiomatic expressions more evenly across layers. |
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections (P19-1)
Copied to clipboard
| Challenge: | Current systems for syntactic analysis tasks rely heavily on large scale annotated data. |
| Approach: | They propose to learn a generative model with a structured prior that uses labeled source and unlabeled target data jointly. |
| Outcome: | The proposed model improves on part-of-speech tagging and dependency parsing tasks on English as the only source corpus and on a wide range of target languages. |
Women’s Syntactic Resilience and Men’s Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing (P19-1)
Copied to clipboard
| Challenge: | linguistic studies have shown the prevalence of various lexical and grammatical patterns in texts authored by a person of a particular gender, but models for part-of-speech tagging and dependency parsing have not adapted to account for these differences. |
| Approach: | They annotate the Wall Street Journal part of the Penn Treebank with the gender information of the articles’ authors and build taggers and parsers trained on this data. |
| Outcome: | The proposed model can account for gendered differences in syntactic tasks and highlight future venues for developing more accurate taggers and parsers. |
Probing LLMs for Joint Encoding of Linguistic Categories (2023.findings-emnlp)
Copied to clipboard
Giulio Starace, Konstantinos Papakostas, Rochelle Choenni, Apostolos Panagiotopoulos, Matteo Rosati, Alina Leidinger, Ekaterina Shutova
| Challenge: | Existing research suggests that a linguistic hierarchy emerges in large language models . little is known about how encodings of different linguistic phenomena interact within the models - and to what extent processing of linguistically-related categories relies on the same, shared model representations. |
| Approach: | They propose a framework for testing the joint encoding of linguistic categories in large language models. |
| Outcome: | The proposed framework shows that the same patterns hold across languages in multilingual LLMs. |
Evaluating distillation methods for data-efficient syntax learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | knowledge distillation (KD) targeting attention should selectively accelerate syntax acquisition, a study finds . logit-based KD dramatically improves data-efficiency, attention-based one provides minimal benefit even for syntactic tasks. |
| Approach: | a study predicts that knowledge distillation targeting attention should selectively accelerate syntax acquisition . a systolic analysis of student models compared to logit-based knowledge distillations . |
| Outcome: | a new study shows that knowledge distillation (KD) targeting attention accelerates syntax acquisition . the hypothesis is tested on syntactic benchmarks and perplexity. |